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Abstract 

In this paper, we introduce new enumerative invariants of curves on Calabi-Yau 3-folds 
via certain stable objects in the derived category of coherent sheaves. We introduce the 
notion of limit stability on the category of perverse coherent sheaves, a subcategory in the 
derived category, and construct the moduli spaces of limit stable objects. We then define the 
counting invariants of limit stable objects using Bchrcnd's constructible functions on that 
moduli spaces. It will turn out that our invariants are generalizations of counting invariants 
of stable pairs introduced by Pandharipande and Thomas. We will also investigate the 
wall-crossing phenomena of our invariants under change of stability conditions. 

1 Introduction 

The purpose of this paper is to introduce new enumerative invariants of curves on Calabi-Yau 
3-folds from certain stable objects in the derived category of coherent sheaves. The notion of 
stability conditions on derived categories, more generally on triangulated categories, is intro- 
duced by Bridgeland [6], motivated by Douglas's work on IT-stability [TT], [12]. However at this 
time, there are some issues in studying Bridgeland's stability conditions on projective Calabi- 
Yau 3-folds. Instead, we consider a generalized notion of stability conditions which we call limit 
stability, and study their stable objects. The limit stability is considered as the "large volume 
limit" in the stringy Kahler moduli space. We construct the moduli spaces of limit stable objects, 
and introduce the enumerative invariants of such objects. 

On the other hand, a kind of enumeration problem of objects in the derived category is 
studied by Pandharipande and Thomas [27], [28J, [29J. We will see how our invariants relate to 
the invariants of stable pairs introduced by them [27]. We will also investigate the wall-crossing 
phenomena of our invariants under change of stability conditions, and propose a conjectural 
wall-crossing formula which is related to the rationality conjecture proposed in [27] . 

1.1 Background 

Let X be a non-singular projective Calabi-Yau 3-fold over C. The Gromov-Witten (GW) in- 
variants of X are counting invariants of curves on X, integrating over the virtual class of the 
moduli space of stable maps M(X), 

M{X) = {(C, /) | /: C — ► X is a stable map from a curve C}. 

Since stable maps have non-trivial automorphisms, M(X) is in general a Deligne-Mumford stack 
and the GW invariants are rational numbers. Another kind of counting invariants of curves on 
X, called Donalds on- Thomas (DT) invariants, are defined as the integration over the virtual 
class of the moduli space of the ideal sheaves, 

I(X) = {I c C O x | C C X is a subscheme with dimC < 1}. 
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Since I(X) is nothing but the Hilbert scheme, the resulting invariants are integer valued. The 
GW-DT correspondence [25] is a conjectural relationship between two generating functions in- 
volving GW invariants, DT invariants respectively. More precisely, one dimensional subschemes 
C C X contain zero dimensional subschemes, hence the DT theory does not directly count 
curves. Instead by dividing by the generating series of counting invariants of zero dimensional 
subschemes, we can define the reduced DT theory which should correspond to the GW theory 
in GW-DT correspondences. 

The notion of stable pairs on X is introduced in [27] in order to give a geometric interpretation 
to the reduced DT theory. By definition a stable pair consists of data (F, s), 

s: O x ^F, 

where F € Coh(X) is a pure one dimensional sheaf, and s is a morphism satisfying the condition 

dimCoker(s) = 0. 

The Pandharipande- Thomas (PT) invariants are defined by the integration over the virtual 
class of the moduli space of stable pairs, 

P(X) = {(F, s) | s : O x — ► F is a stable pair }, 

and the DT-PT correspondence [27] is a conjectural relationship between generating functions 
of reduced DT theory, PT theory respectively. 

On the DT-side, any ideal sheaf Iq C Ox is a Gieseker-stable sheaf, hence DT-invariants 
count stable objects in Coh(X). On the other hand, the space P{X) can be viewed as the 
moduli space of the two term complexes, 

/' = {Ox —> F} G D b (X), 

where D b (X) is the bounded derived category of coherent sheaves on X. Furthermore the 
obstruction theory which admits the virtual class on P{X) is obtained from the deformation 
theory of objects in D b (X), not from that of stable pairs. From this observation, we guess 
that 7* might be stable objects with respect to a certain stability condition on D b (X), and 
PT-invariants count stable objects. 

Now we are led to consider stability conditions on D b (X), and enumerative problem of stable 
objects in D b (X). In the next paragraph, we discuss stability conditions on derived categories. 

1.2 Stability conditions on triangulated categories 

Let T> be a triangulated category, e.g. T> = D b (X) for a smooth projective variety X. The 
notion of stability conditions on T> is introduced by Bridgeland [6]. Roughly speaking a stability 
condition on V consists of data a = (Z,A), 

Z:K(V) — >C, AcV, 

where Z is a group homomorphism called a stability function, and A is the heart of a bounded 
t-structure on T>, which satisfy some axiom. When T> = D b (X), the set of locally finite numerical 
stability conditions Stab(X) is shown to have the complex structure by Bridgeland [6J, and the 
quotient space 

Auteq(£>)\Stab(X)/C 
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is a mathematical candidate of the stringy Kahler moduli space. The space Stab(X) have been 
studied in several examples. For instance see [7J, [8], [9], [T7], [24] > [30] , [31], |32j. 

Although the notion of stability conditions on triangulated categories has drawn much in- 
terest recently, we are not able to study the most important case, T> = D b (X) for a projective 
Calabi-Yau 3-fold X at this time. In this case, there are some technical difficulties to construct 
examples of stability conditions, so we do not know whether Stab(X) is non-empty or not. From 
the ideas in physical articles [TT], |12| . there should exist stability conditions corresponding to 
the neighborhood of the large volume limits, whose stability functions are given by, 

Z a (E) = - f e-( B+ ^ch(£)ytd^, (1) 

where a = B + iu G H 2 (X,C) with u an ample class. Such stability conditions should be 
parameterized by elements of the complexified ample cone, 

a G A(X)c = {B + iu G H 2 (X, C) | u is an ample class}. 

Instead of working with Bridgeland's stability conditions, we introduce and study a generalized 
notion of stability conditions which we call limit stability. The corresponding heart of a t- 
structure is the category of perverse coherent sheaves, 

A p C D b (X), 

in the sense of Bezrukavnikov [5 J and Kashiwara |21j . We will see that for a G A(X)c, the 
stability function (jlip together with taking uj — > oo determines the set of (semi) stable objects in 
A p , which we call a-limit (semi) stable objects. The notation "limit" is used to emphasize that our 
stability conditions should correspond to the limit point u = oo. Some fundamental properties 
of limit stability (e.g. existence of Harder-Narasimhan nitrations, Jordan-Holder nitrations,) 
will be studied in Section [2j 

1.3 Main results 

We shall study the enumerative problem of cr-limit stable objects E G AP. Let us take (3 G 
H 4 (X, Q) and n G Q. We first show the existence of the moduli space of limit stable objects. 
The following theorem will be shown in Section [3l 

Theorem 1.1. There is a separated algebraic space of finite type £°(X,[3), which parameterizes 
a-limit stable objects E G A p , satisfying det E = Ox and the following numerical condition, 

(ch (£),chi(£),ch 2 (E),ch 3 (£)) = (-1,0, An) G H*(X,Q). 

It will turn out that the moduli space C^(X, (3) could be non-empty only if (3 is the Poincare 
dual of the homology class of an effective one cycle on X, and n G Z. (cf. Remark 13.31 ) By 
Theorem 11.11 and using Behrend's constructible function [4J, ul - C^(X,(3) — > Z, we are able to 
define the counting invariant of limit stable objects, 

We next show the relationship between the integers L n ^(a) and P n 8, where P n ,p is the PT- 
invariant counting stable pairs (F, s) with 

ch 2 (F) = 0, ch 3 (F) = n. 

See Definition 14.41 for the detail. We show the following theorem in Section [4} 
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Theorem 1.2. Let a = ku> + iuj for k € R. We have, 

L n ,p( a ) = p n,p, (A;<0), L^(o-) = P_ ni/3 , (/c»0). 

It seems that Theorem 11.21 is related to the rationality conjecture of the generating function 
of the PT-invariants, 

It is proposed by Pandharipande and Thomas in [271 Conjecture 3.2] and they conjecture that 
Zp T (q) is a rational function of q, invariant under q i— » 1/g. This conjecture is solved when (3 is 
an irreducible curve class in [29] by comparing P n ^ and P- n ,i3- I n the following, we propose a 
conjectural wall-crossing formula of our invariants L n ^(cr), which combined with Theorem 11.21 
provides a relationship between P n p and P- n ,/3 hi a general situation. For /j, G Q, let feo = — /i/2 
and k_ < ko, k + > ko are sufficiently close to &o- We set <r* = /c*u; + iuj for * = 0, ±. 

Conjecture 1.3. There is a virtual counting of one dimensional uj-Gieseker semistable sheaves 
F with (cli2(-F), cli3(F)) = (f3',n'), denoted by N n i^> G Q, such that 

L ni/3 (a-) - L n>l3 (a + ) = ^{-l) n '~ l n' N n , ^L nll ^{gq). (2) 

Here in the above sum, (/?', n'), n") must satisfy (3' + (3" = (3, n' + n" = n and n' /u)(3' = fi. 

See Paragraph 14.31 for the explanation of the above conjecture. In Section [5l we investigate 
the wall-crossing phenomena of limit stable objects and study Conjecture [L3] in some examples. 
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1.5 Notation and convention 

We work over varieties over C. For a variety X, we denote by D b (X), K(X) the bounded derived 
category of coherent sheaves on X, the Grothendieck group of coherent sheaves respectively. For 
a triangulated category V and a set of subobjects S C V, we denote by (S) C V the smallest 
extension closed subcategory which contains S. If S is a set of subobjects in an abelian category 
A, we also use the same notation (S) C A. 

2 Limit stability 

2.1 Bridgeland's stability conditions 

Here we briefly review the definition of Bridgeland's stability conditions [6]. Let us begin with 
the stability conditions on abelian categories. 
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Definition 2.1. |6j Let A be an abelian category. A stability function on A is a group homo- 
morphism, 

Z: K(A) — ► C, 
such that for any non-zero E G A, we have 

Z(E) £i:= {rexp(i7T0) | r > 0,0 < < 1}. 

Given a non-zero object E £ A and a stability function Z : if (.4.) — > C, we can uniquely 
determine the phase of E by 

cj)(E) = -Imlag Z(E) e (0,1]. 

7T 

We say E € .4 is Z-semistable if for any non-zero subobject E C E in ^4, we have 

4>(F) < 4(E). 

Definition 2.2. |6j A stability function Z : K(A) — > C is called a stability condition on „4 if for 
any E £ A, there exists a filtration 

= E C E 1 C ■■■ C E n = E, 
such that each Ej = Ej/Ej_i is Z-semistable with 

0(F x )>0(F 2 ) > ••• >0(E n ). 
The above filtration is called a Harder- Narasimhan filtration. 

It is easy to construct examples of stability conditions if A has finite number of simple objects 
Si, ■ ■ ■ , Sjv E A such that 

*4 = (Si, ■ ■ ■ , Sn). 

e.g. A = modj4 for a finite dimensional /c-algebra A. In this case K(A) is generated by 
[Sy E ET(*4), and Z: i^T(^t) — > C is a stability condition if and only if Z(S{) € HI for all i. 

In general, a sufficient condition for a stability function to be a stability condition is provided 
in [6l Proposition 2.4]. 

Proposition 2.3. [HI Proposition 2.4] Eei Z: ET(»4.) — > C 5e a stability function. Assume 
that 

• there is no infinite sequence of inclusions in A, 

■ ■ ■ ^ E n ^ ■ ■ ■ <^-v Ei <^-> .Eo) (3) 
with 4>(Ei + i) > <t>(Ei) for all i. 

• there is no infinite sequence of surjections in A, 

E ^> Ei -» » E n -» • ■ • , (4) 

with 4>(Ei) > 4>(Ei + i) for all i. 
Then Z is a stability condition. 
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Next let T> be a triangulated category, e.g. V = D (X) for a variety X. The following is the 
definition of Bridgeland's stability conditions. 

Definition 2.4. |6j A stability condition on T> consists of data (Z,A), where A C T> is the 
heart of a bounded t-structure on T>, and Z is a stability condition on .A. 

Remark 2.5. A stability condition on D in [6] is originally given by data (Z,V), where 
Z : K(T>) — > C is a group homomorphism, and V(4>) C X> for (/> E M are full subcategories, 
satisfying some axioms. However as shown in [BJ Proposition 4.2], this is equivalent to giving 
data (Z,A) as in Definition 12.41 

Remark 2.6. Let Z : K(A) — > C be a stability condition on an abelian category A. Then the 
pair (Z,A) is a stability condition on the triangulated category D b (A). 

Let 2? = D b (X) for a smooth projective variety. A stability condition (Z,A) on V is called 
numerical if -KX-X") — > C factors through the Chern character map, 

ch 

#*(X,Q) 

The set of numerical stability conditions onP = D b (AT) which satisfy the local finiteness (cf. [6j 
Definition 5.7]) is denoted by Stab(X). In [BJ Theorem 1.2], Bridgeland shows that Stab(X) has 
a structure of a complex manifold. 

The space Stab(X) is studied when dim A" = 1, 2 in the articles [BJ, [7]. Unfortunately, we do 
not know how to construct examples of stability conditions for higher dimensional varieties. It 
seems that the following lemma is well-known, but we put it to emphasize that the construction 
problem is non-trivial. 

Lemma 2.7. Let X be a smooth projective variety with d = dimX > 2. Then there is no 
numerical stability condition (Z,A) on D b (X) with A = Coh(X). 

Proof. It is enough to show that there is no stability function Z : K(X) — > C on Coh(A") of the 
following form, 

d 

Z(E)=J2(u j + iv j )ch j (E), 
i=o 

for Uj + ivj € H 2d ~ 2 i(X, C). Suppose that such a stability function Z exists. Since d > 2, there 
is a smooth subvariety S X with dimS" = 2. Then the composition 

K(S) K(X) C, 

is a stability function on Coh(S'), hence we may assume d = 2. Let C C X be a smooth curve 
and take a divisor D on C. Since Im Z(E) > for any E G Coh(X), we have 

lmZ(O c (D)) = v 2 (degD + ch 2 (O c )) + v x ■ [C] > 0. 

Since we can take D with an arbitrary degree, we must have v 2 = 0. Similarly we have 

Im Z(O x {mC)) = m«i • [C] + v > 0, 
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for any m G Z, hence v\ ■ [C] = 0. Therefore Im Z(Oc(D)) = 0, and this implies 

Re Z(O c (D)) = u 2 (deg D + ch 2 (O c )) + «i • [C] < 0, 
since Z(0<7(D)) € HI. Then the same argument shows that u 2 = 0, and this implies 

Z(O x ) =u 2 + iv 2 = 0, 

for any closed point This contradicts that Z(O x ) G H. □ 

Remark 2.8. In the case of dimX = 2, the examples of stability conditions (Z,A) are con- 
structed in [7], [2] by setting A to be the tilting of Coh(X) with respect to certain torsion pairs. 
When X is a K3 surface, the stability function Z is given by 

Z{B^){E) = - J e~( B+ ^ ch(E)Vtd^, 

for B + iuj G H 2 (X,C) with a; an ample class. When dimX > 3 we expect that for to ^> 0, 
there are hearts of bounded t-structures Am,u) such that the pairs {Zi B ^)->Am,w)) determine 
stability conditions, giving the neighborhood of the large volume limits. However at this time, 
we are not able to find such Am,uj)- 

2.2 Perverse coherent sheaves on Calabi-Yau 3-folds 

From this paragraph, we focus on the case that X is a Calabi-Yau 3-fold, i.e. X is a smooth 
projective 3-fold with a trivial canonical class. Here we study the heart of a bounded t-structure 
A p , constructed as one of the perverse t-structures introduced by Bezrukavnikov [5] and Kashi- 
wara |21j. In the notation of Remark 12.81 the desired category Am,u) should be constructed 
as an approximation of our category A p , so we hope that studying A p in detail will solve the 
construction problem in a future. Let us recall the notion of torsion pairs and their tilting for 
the construction of A p . 

Definition 2.9. Let A be an abelian category. A torsion pair on A is a pair of full subcategories 
(T, J-) such that 

• For T € T and F £ J-, we have Hom(T, F) = 0. 

• For any E E A, there is an exact sequence 0— >T— > E — ^F^Oin^l such that T € T, 
F G T. 

Given a torsion pair (T,!F) on A, the following subcategory of D b (A), 

= {E £ D b {A) I H~ l {E) € F,n°{E) G T,1-e{E) = f or i + -1,0}, 

is known to be the heart of a bounded t-structure on D b (A), and it is called a tilting with respect 
to the torsion pair (T, J r ). (cf. [E].) For a Calabi-Yau 3-fold X, we have the following torsion 
pair. 

Lemma 2.10. The pair (Coh<i(X), Coh> 2 (X)), 

Coh<!(X) := {E G Coh(X) [ dimSupp(£) < 1}, 

Coh> 2 (X) := {E G Coh(X) I Hom(F, E) = for any F G Coh<i(X)}, 
is a torsion pair of Coh(X). 
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Proof. For an object E G Coh(X) there is an exact sequence, 



— ► T — > E — > F — ► 0, 

such that T G Coh<i(A) and dim Supp(i ? ) > 2. Since Coh(X) is a noetherian abelian category, 
we can take T to be maximum, i.e. there is no T' G Coh<i(A) with T C T' C £7. Then it is 
easy to see that F G Coh>2(X). □ 

Our abelian category .4 P is constructed tilting. 

Definition 2.11. We define the Ziearf of a perverse t-structure A p C D b (X) to be the tilting 
with respect to the torsion pair (Coh<i(X), Coh>2(A)), i.e. 

.4P = (C!oh> 2 (X)[l],Coh<i(X)>. 

Remark 2.12. In general, a perverse t-structure introduced in [5], [21] is determined by 
choosing a perversity function, which is a map p: X top — ► Z satisfying a certain condition. One 
can easily check that our category A p corresponds to the following perversity function, 



p{x) 



-1 dimO XjX <l, 
dimO Xa; >2. 



Remark 2.13. The subcategory Coh<i(X) C A p is easily seen to be closed under quotients 
and subobjects, hence it is an abelian subcategory. Since Coh<i(X) is not artinian, the abelian 
category A v is also not artinian. 

Remark 2.14. The abelian category A v is also not noetherian. In fact let us take a divisor 
H C X and a curve C C H. Then there exists an infinite chain of surjections in A p , 

O h [1] -» O h (C)[1] -» O h {2C)[1] -» ■ ■ ■ . 
2.3 Torsion pair on AP and the dualizing functor 

As we have seen in Remark 12.141 the abelian category A p is worse than Coh(X), and this fact 
sometimes causes difficulty to handle A p . In this paragraph, we introduce a certain torsion pair 
on A p which makes A p much more amenable. Let us set A\, Ay 2 to be the subcategories of A p , 

A\ '■= (F[l), O x | F is a pure two dimensional sheaf and x £ X), 
A p /2 := {E € A p | Hom(F, E) = for any F € A 9 }. 

The meaning of the subscript will be clear in Lemma 12,261 

Remark 2.15. For E G A p , it is obvious that E G A\ if and only if H°(E) is zero dimensional 
and TL~ l {E) is a torsion sheaf. Also E G Ay 2 ^ an£ i only if 7i^ 1 (E) is torsion free and 
Hom(0 x ,.E) = for any x G X. In particular, we have 

Ay 2 H Coh<i(A) = {pure one dimensional sheaves}. 

We show the following lemma. 

Lemma 2.16. The pair (A^, A^ 2 ) i s a torsion pair of A p . 
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Proof. It is enough to show that for any E G A p , there is an exact sequence in A p 

— ► Ei — > E — > E l/2 — » 0, 

with Ei G A\ for i = 1, 1/2. Let F C Tl~ l (E) be the maximum torsion subsheaf. We have the 
exact sequence in A p , 

— ► F[l] — > E — > E' — ► 0. 

Note that F[l] G A\ and Ti~ 1 (E') = Ti~ l {E)/F is torsion free. Hence for any pure two 
dimensional sheaf F', we have 

Hom(F'[l],£') = Rom(F',H~ 1 (E')) =0. 

Therefore if E' is not contained in A\ ; 2 j there is a zero dimensional sheafs such that Hom(ZY, £") 7^ 
0. By Remark I2,13[ this means that there is a subobject W C E' in such that W is a zero 
dimensional sheaf. Moreover we can take W C E to be maximum, i.e. there is no zero dimen- 
sional sheaf U" with U' C C £7' in ^4 P . To show this, it is enough to check that any sequence 
of subobjects, 

U x C U 2 C • • • C U n C • • • C E', (5) 

where Ui are zero dimensional sheaves, terminates. Let G{ = E' /Hi G A p . We have the exact 
sequence in A p , 

— ► Ui/Ui +1 -^G l ^ Gi+i — » 0. 
Taking cohomology and noting that 7i~ l {E') is torsion free, we see that 

TT 1 ^') c H _1 (6?i) c H- 1 (G 2 ) c • • • c TT^Gy c ■ ■ ■ c H-\E') VV , (6) 

in Coh(X). The sequence (J6|) must terminate, say H~ 1 (Gj) = H~ 1 (Gj + i) = Replacing 
E' by Gj, we may assume that TH. {E') = 7i~ 1 (Gi) for any i. Then each are subsheaves of 
TiP(E'), thus ([5]) must terminate. Therefore there is a maximum zero dimensional sheaf W C 
Now let E" = E' /W , and consider the exact sequences in A p , 

— ► F' — > E — ► E" — ► 0, 
— ► F[l] — > F' — >W — ► 0. 

Here E —» E" is obtained as the composition of the quotients in A p , E — » E' -» E" . The bottom 
sequence shows F' G A\. By the construction, we also have E" 6 Ay 2 . '-' 

Let D: D b {X) -> Z) b (X) op be the dualizing functor, 

B(£) = RHom(S,O x [2]). 

In the following lemma, we see the compatibility of the torsion pair {A\, Ay 2 ) with the dualizing 
functor D. 

Lemma 2.17. We have 

EeA{=>B(E) eA^-l], 

E G A\ j2 => D(£) G ^ /2 . 
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Proof. First we show that B(E) G -A?[-l] for E G A p v It is enough to check this for E = G[l] 
and E = O x , where G is a pure two dimensional sheaf and x G X is a closed point. Since G is 
pure, we have 

Sxt^(G,O x ) = 0, fori 7^1, 

and £xt x (G,Ox) is a pure two dimensional sheaf, (cf. [144 Section 1.1].) Therefore B(G[1]) G 
^[-1]. Also we have 0(0*) = 0^-1] £ 

Next let us take E 1 G an< ^ cnec k D(i?) G «4^/ 2 - Since H°(E) is a torsion sheaf and £7 is 
concentrated on [-1, 0], we can easily see H l (P>(E)) = for i < -2. Suppose that H k {B(E)) ^ 
and Tt' l (H)(E)) = for any i < k. Then there is a closed point x G X such that 

/ Rom(B(E),O x [-k}) = Rom(O x [k -1},E). 

Therefore we have k < 0, and 0(£?) is concentrated on [—1,0]. Let us take F G Coh<i(X). 
Since we have 

W*(D(F[1])) =0 for i < 0, 

it follows that 

Hom(F,7i" 1 (B(E))) = Horn (F[l], 0(E)) 
= Hom(£,ID)(F[l])) 
= 0. 

Hence 7^~ 1 (D(E')) G Coh>2(X). Let us take a codimension one point p G X. Since 7l! _1 (S) 
is torsion free, we have E p = 0^ r p [l] for some r. Therefore B(£ , ) p = 0^ r p [l], and this implies 
H°{B(E)) G Coh<i(X), i.e. B(E) G *4 P . Moreover for any object E' G A{, we have 

Hom^',^) Hom(D(£),B(£')) = 0, 

since O^') G ^[-1]. Therefore we can conclude 0(£?) G ^ /2 . □ 

Remark 2.18. According to [21], the abelian category D(„4 P ) corresponds to the heart of a per- 
verse t-structure A p (up to shift) with the dual perversity function p* : X top — > Z. Lemma [2. 171 
implies that „4 P is obtained as a tilting with respect to the torsion pair (A p , Ay 2 ). 

Let us take E,F G and a morphism f:E — > F. The morphism / is called a strict 
monomorphism if / is injective in and Coker(/) G A\. Similarly / is called a strict epimor- 
phism if / is surjective in A p and Ker(/) G «4.f. Although the category A p is not artinian nor 
noetherian, each subcategories A p have such properties. 

Lemma 2.19. Fori = 1, 1/2, the category A\ is of finite length with respect to strict monomor- 
phisms, and strict epimorphisms, i.e. any infinite chains of strict monomorphisms, strict epi- 
morphisms in A\, 

■ • • ^ E n ^-y ■ ■ ■ ^-y Ei <^-> Eq, (7) 
E Q -» Ei -» (8) 

must terminate. 
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Proof. By applying the dualizing functor D and using Lemma 12.171 h is enough to show that a 
chain ([7]) terminates. Let us take an infinite chain (JT]) in A\ with each Ei G A\. Let ui be an 
ample divisor on X. Since — chi(E) ■ oj 2 > for E G A\, we have 

-chi(Ei) -u? > -chi^i+i) • uj 2 > 0. 

Hence we may assume that chi(E'j) • to 2 = chi(£'j + i) • to 2 for any i, and this implies that the 
induced morphism 

H-\Ei) — ►TCH^+i), 
is an isomorphism in codimension one. Let us take the exact sequence in A p , 

O^Ei^ E i+1 ^G l ^0. (9) 

Then 7^ _1 (Gj) = since otherwise TL~ l {G{) is one or zero dimensional, and contradicts that 
TL~ l {Gi) G Coh>2(X). Taking the cohomology of ([9|), we have the chain of inclusions of sheaves, 

• • • C H\E n ) C • • • C H Q {Ei) C H°(E ). (10) 

The sequence (|10p must terminate since each TC°(Ej) is a zero dimensional sheaf by the definition 
of A\. Hence the chain ([7]) also terminates. 

Similarly let us take a chain ((7J with each Ei G Ay 2 . Then we have 

-cho(^) > -ch (^+i) >0, 

hence we may assume — eho(-Ej) = — cho(Sj+i) for any i. Let us consider the exact sequence as 
in ©. Again Ti^ l {G-i) = since otherwise it is a two dimensional sheaf, which contradicts that 
Gi G Ay 2 - Taking the cohomology of (JI3), we obtain the sequence (fTOl) . In this case, we have 

ch 2 (W°( J E i )) • uj > ch 2 (H°(E l+1 )) -u>0, 

hence we may assume ch2(W° {Ei))-u = ch2('H (Ei + i))-u>. Then Gi = H°(Gi) is zero dimensional, 
thus Gi = by the definition of Ay 2 - Therefore dTJ must terminate. □ 

2.4 Limit stability on A p 

Here we introduce the notion of limit stability on A p . Let A(X)c be the complexified ample 
cone, 

A(X)c := {B + iui G H 2 (X,C) \ uj is an ample class }. 
For a = B + iuj G A(X)c, we consider the group homomorphism Z a : K(X) — > C, 

Z a (E) = - J e -( fi +-) ch(^)ytd^. (11) 

The above function does not give a stability function on A p . However if we replace a by 

a m = B + mioj, for m S> 0, 

then we can define the well-defined argument of Z am (E) for any non-zero E G A p , which defines 
the set of (semi)-stable objects in A p . To see this in more detail, let us introduce the (twisted) 
Mukai vector, 

v B : K{X) 3 Ei — > e- B ch(E)^/td^ G H*(X,R). 
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Let v B (E) & H 2l (X,~R) be the i? 2i -component of v B (E). Then one can expand (fTTj) and give 
the following formula, 



Z« m {E) = - Je-™»v B {E) (12) 
-v£(E) + ^m 2 uj 2 v B (E) \ + (muJV B (E) - ^m 3 u 3 v B (E)\ i. (13) 
We have the following lemma. 

Lemma 2.20. For a non-zero object E € A p , we have 



jrexp(i7T0) : r > 0, ^ < < ^| , 



Z am {E) e { rexp(m<f>) : r > 0, 7 < < - \ , (14) 
/or m 3> 0. 

Proof. Let us take E" 6 Coh(X) with dimSupp(-E) = 3 — i for < i < 3. It is easy to see that 
v B (E) = for j < i and 

wf (E) ■ oj 3 ~ 1 = chi(E) ■ cu 3 - 1 > 0. 
Therefore by the formula (|13j) . the argument of Z Gm (E) for m — > oo goes, (modulo 27r, ) 



arg^jE) 



7r dim Supp(E') = 0, 
f dimSupp(-E) = 1, 



dimSupp(£) =2, ^ 15 ^ 



V 2 



dim Supp(E') = 3. 



Since the category AP is generated by Coh<i(X) and Coh>2(X)[l], the above asymptotic be- 
havior of Z am (E) shows the result. □ 

Given a £ A(X)c and a non-zero object E £ .A p , we can uniquely determine the phase of 
Z* m (E) by 

<^ m (£) = 3lmlogZ CTm (£)€ " ' 



"> A \ "I 



7T ° m V 7 V 4 4 

for m ^> 0. For non-zero F,E G „4 P , we simply write 

4>(t{f) -< ME), MF) d 4>a(E), 

if ^a^F) < 4> am (E), 4>a m {F) < <p am (E) for m S> respectively. Below we introduce the notion 
of limit (semi) stable objects. 

Definition 2.21. For <r € A(X)c, a non-zero object € .4 P is called a-limit stable (resp. 
a-limit semistable) if for any non-zero subobject F C E, one has 

CT (F) ~< ME), (resp. <f> a {F) < ME)-) 

Remark 2.22. In Lemma 12.201 the smallest m > for which (|14|) holds depends on E, the 
function Z am does not give stability functions on A p for any m. On the other hand, the function 
Z (j induces the stability condition on the subcategory Coh<i(X) C A v by the composition, 

K(Coh< x (X)) — ► K(X) C. 
The induced stability condition is the same one constructed in [34|, Lemma 3.4]. 
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Remark 2.23. Our notion of limit stability is included in the notion of polynomial stabil- 
ity introduced by A. Bayer [3] independently. Some of the results in this section, especially 
Theorem 12.291 (i) . are proved in [3] in more general setting, although the proofs are different. 

Remark 2.24. It is easy to see some standard stability properties for limit stability. For 
example, let E,F £ A p be cr-limit semistable with (p a {E) y (p a {F). Then Hom(£, F) = 0. Also 
for cr-limit stable object E £ A p , one has Hom(£', E) = C. 

In the following, we give some examples of limit (semi)stable objects. The proofs are straight- 
forward and we leave them to the readers. 

Example 2.25. (i) Let F be a //-stable vector bundle on X. Then F[l] £ A p and it is cr-limit 
stable for any a £ A{X)c- 

(ii) Let us take a = B + iuj £ A{X) C and F £ Coh<i(A) C A p . Then noting Remark 
and Remark l2.22l we can easily see that F is a cr-limit semistable if and only if F is (B, a>)-twisted 
semistable sheaf, i.e. for any non-zero subsheaf F' C F, one has n a (F') < fj, a {F), where 

uJch 2 {F) 

(iii) Let x £ X be a closed point and I x C Ox the ideal sheaf. Then I x is a Gieseker stable 
sheaf, but I x \\\ S A p is not cr-limit semistable. In fact we have the exact sequence in A p , 

0^O x ^ I x [l] — O x [l] — » 0, 

with (pa {O x ) >- (p a {I x [i\), which destabilizes I x [l]. 

For objects in A p , we have the following lemma. 

Lemma 2.26. For a non-zero object E £ A- (i=l, 1/2,) we have 

0<x m form^oo. 

Proof. For E = F[l] or E = O x , where F is a pure two dimensional sheaf and x £ X is a closed 
point, the result follows by the formula (|13p . By the definition of .4?, the result follows for any 
E £ Next let us take a non-zero object E £ .A^- Then by the definition of Ay 2 , we have 
either H~ 1 {E) is a torsion free sheaf, or Ti 1 {E) = and Ti.°{E) is a pure one dimensional sheaf. 
In both cases, the result follows by the formula (|13p . □ 

We have the following characterization of limit stable objects. 

Lemma 2.27. An object E £ A p is a -limit (semi) stable with (p Um {E) —> i for m — > oo if and 
only if E £ A\ and for any strict monomorphism ^ F E in A\, one has <p a {F) -< (p a {E). 
(resp. 4> a {F) ■< 4> a {E).) 

Proof. Suppose first that E is cr-limit semistable. By Lemma |2.16| there is an exact sequence, 

_ Ei — ► E — » E 1/2 — » 0, 

in ^4 P with £"j £ Ai- By Lemma |2.26[ the limit semistability of E implies E\ = or -E1/2 = 0. 
Hence if (p am {E) goes to i, we have E £ ^4?. Next assume that E £ .4^ and consider an exact 
sequence in A p , 

— > F — ► E — > G — >• 0. 
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By Lemma f2.16( there is an exact sequence — * F\ — > F — > F^ 
yields, 

<M*i) >: <MF) Mifya)- 
Composing the injections Fi F F, we obtain the exact sequence in „4 P , 



with Fj G A P . Lemma ESS 







-Pi 



F 



a 



0. 



For any F' G .A? we have Hom(G",F') C Hom(F, F') = 0. Therefore G' G A?, i.e. Fi ^ F 
is a strict monomorphism. Hence if <f) a (F\) H (j) a {E) holds, then <j) a {F) H CT (F), hence F is 
(T-limit semistable. The proofs for limit stable objects and the case of i = 1/2 are similar and 
we leave them to the reader. □ 

For a = B + iu G A(X)c, let <7 V = — F + iw. Combining Lemma 12.171 with Lemma 12.271 we 
have the following compatibility of limit stability with the dualizing functor. 

Lemma 2.28. We have the following. 

E G A\ is a -limit (semi)stable B(F)[1] G A\ is a y -limit (semi)stable, 
E G Ay 2 is a-limit (semi)stable ^ B(F) G A^ 2 is a v -limit (semi)stable. 

Proof. For v G H cvcn (X,R), let v v be the dual operator, 

v = (v ,vi,v 2 ,v 3 ) i — > v y = (v , -vt,v 2 , -v 3 ). 

Here v i is the F 2 *-component of v. Then we have v- B (B(E)) = v B (E) v , hence 



Z a v(B(E)) 



-miu) „,B 



v u (Ef 



= -Z C {E). 

Therefore the result follows from Lemma 12.171 with Lemma 12.271 directly. 

Finally in this section, we prove the following theorem. 

Theorem 2.29. For a G A{X)^, we have the following. 

(i) For a non-zero E G A v , there exists a filtration in AP , 



□ 



Fn C Fi C • • • C E n 



E, 



such that each Fj = Fj/Fj + i is a-limit semistable with CT (Fj) y ^(Fj+i) 
Harder- Narasimhan filtration. 

(ii) For a a-limit semistable object E G A p , there exists a filtration in A p , 



Eq C Fi C • • • C E n — F, 

such that each Fi = Ei/Ei + \ is a-limit stable with 4> a {Fi) = 
Holder filtration. 



(16) 

i.e. [To]) is a 
(17) 



i+l, 



i.e. 



Til) is a Jordan- 
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Proof, (i) In Proposition 12.31 let us replace A, (ft, "inclusions", "surjections" , by A p , (j) a , "strict 
monomorphisms" , "strict epimorphisms" , respectively. As Lemma [2.19l provides the correspond- 
ing sufficient condition, we can follow the same proof of Proposition 12.31 in ^\ Proposition 2.4], 
and show the following. For any E G A?, there is a finite sequence of strict monomorphisms in 
A p 

Eq Ei w ■ ■ ■ <^-> E n = E, (18) 

such that for any strict monomorphism F <^-> F$ = Ei/Ei + \ G A p , one has 4> a (F) ^ <f>cr(Fi), an d 
(t>a{Fi) y (j) a (Fi + i). By Lemma T2.27[ Fj is a u-limit semistable object, hence the filtration (fT5|) 
gives the Harder-Narasimhan filtration. 

Let us take an object E G A p . We have an exact sequence, 

— > Ei — > E — > Ei /2 — » 0, 

with Ei G A P . Composing the Harder-Narasimhan filtrations of Ei, Ei/ 2 , we obtain the Harder- 
Narasimhan filtration of E. 

(ii) Since any u-limit semistable object is contained in A\ or A p , 2 by LemmaE271 the result 
follows from Lemma 12.191 □ 

Remark 2.30. The existence of a Harder-Narasimhan filtration is guaranteed once we show 
that there are no infinite sequences such as ([3j), (|4]) for limit stability. Unfortunately this is not 
true. In fact in the notation of Remark 12.141 we have the following infinite sequence, 

Ox e o H [i] - Ox e o H (c)[i] ^o x ® o H {2C)[i] 

which satisfies that for a = iu, 

MOx e O h [1\) y MOx e O h {C)[1\) y MOx O h (2C)[1\) y ■ ■ ■ . 

3 Moduli spaces of limit stable objects 

For a Calabi-Yau 3-fold X, let us take elements, 

/?G# 4 (X,Q), r*€# 6 (X,Q) ^Q. 

This section is devoted to study the moduli problem of limit stable objects E G A p , satisfying 
det E = Ox and the following numerical condition, 

(ch (E),chi(E),di 2 (E),ch 3 (E)) = (-1,0, /?,n). (19) 

Note that if E G A p is limit stable satisfying (fTSJ), then E G A\, 2 by Lemma E23 For 
a G A(X)c, let L n (X,f3), L°(X,0) be the sets objects, 

L n (X,P) := {E G A P 1/2 | detE = O x and ch(E) satisfies (O }, (20) 
L°(X, 0) '■ = {E G L n (X, /?) | £ is a-limit stable }. (21) 

Remark 3.1. If E is quasi-isomorphic to a two term complex {Ox F), where F is a pure 
one dimensional sheaf located in degree zero, and 

ch 2 (F) = 0, ch 3 (F) = n, 

then E G L n (X,0). 
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From this section, we use the following notation. For a relatively perfect object (cf. [231 
Definition 2.1.1]) £ G D b (X x S) and a morphism T -> S, we denote by £ T G D b (X x T) the 
derived pull-back of £. The moduli problem of objects in the derived category has been studied 
in some articles, see [IS] . [23] . [33] , In this paper, we use the algebraic space constructed by 
Inaba [16J, which provides a "mother space" of our moduli problem. Let hA be the functor, 

M: (Sch/C) — > (Set), 

which sends a C-scheme S to a family of simple complexes £ G D b (X x S 1 ), (up to isomorphism,) 
where an object E G D b {X) is called a simple complex if 

Hom(£, £) = C, Ext -1 (S, E) = 0. (22) 

Then Inaba [16] shows that the etale sheafication of ftA, denoted by A4 et , is an algebraic space 
of locally finite type. Let M.^ 1 be the closed fiber at [Ox] G Pic(X) with respect to the following 
morphism, 

det: M et 3£i — > det E G Pie(X). 
Since any object E G L°(X,[3) satisfies (f2"2"|) . there is a sub functor 

CM§\ (23) 

whose S'-valued point consists of £ G M.fftS) with £ s G £^(^, /?) for any s G 5. Our purpose in 
this section is to show that C^{X, /3) is an algebraic subspace of . We use the same strategy 
as in [55] . namely we show that (|23|) is an open immersion and L°(X,f3) is bounded. 



3.1 Characterizations of limit stable objects 

In this paragraph, we give some characterizations for objects in L n {X,(3) to be limit stable. 
First we show the following. 

Lemma 3.2. For an object E G L n (X,f3), there is a subscheme C C X with Oc a pure one 
dimensional sheaf (or zero) such that 7i~ l (E) is isomorphic to the ideal sheaf I c C Ox- 

Proof. Since E G -4^/2' TC~ 1 (E) is a torsion free sheaf of rank one with trivial determinant. We 
have the injection, 

n- 1 (E)^n- 1 (E) vv ^o x , 

which shows TC~ 1 (E) = Iq for a subscheme C C X. By the condition (I19p . we have dimC < 1. 
Also if Oc contains a zero dimensional subsheaf, there is x G X with injections in A p , 

O x ^O c ^ I c [l] ^ E. 

Here Oc Ic\\] corresponds to the extension — > Ic — > Ox — * Oc — > 0. Since E G Ay 2 , this 
is a contradiction. □ 

Remark 3.3. For E G L n (X,f3), Lemma E2] yields, 

/3 = ch 2 (O c ) + ch 2 (H°(E)) G tf 4 (X, Z), (24) 
n = ch 3 (O c ) + di 3 (H°(E)) G ^ 6 (X, Z) = Z. 

Hence below we always assume /? G H 4 (X, Z), n G Z, and /9 is an effective class, i.e. the Poincare 
dual of the homology class of an effective one cycle on X. 
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Next we show the following. 

Lemma 3.4. For a E A(X)c, an object E E L n {X,(3) is a -limit stable if and only if the 
following conditions hold. 

(a) For any pure one dimensional sheaf G / which admits a strict epimorphism E — » G 
in A\i 2 , one has (p a (E) -< <f> G (G) . 

(b ) For any pure one dimensional sheaf F ^ which admits a strict monomorphism F E 
in Ay 2 , one has <fi(j(F) -< 4><j{E). 

Proof. For a cr-limit stable object E E L n (X,/3), the conditions (a), (6) follow from the definition 
of limit stability. 

Next suppose that E E L n (X,f3) satisfies (a) and (b). Applying Lemma T2.271 it is enough to 
show that for any non-trivial exact sequence in A^, 2 , 

— > F — > E — >G — >0, 

we have (j> a (F) -< 4> a (E). If 7i^ 1 (F) = 0, then F is a pure one dimensional sheaf and 4> a {F) -( 
(j)(j{E) follows from (b). If Tl~ l {F) ^ 0, then it is a torsion free sheaf of rank one by Lemma 13.21 
It follows that Ti (G) is a torsion sheaf, hence zero because of G E Ay 2 - So G is a pure one 
dimensional sheaf, and we obtain <p <T {E) ~< 4> a {G) by (a). □ 

Remark 3.5. By Lemma 12.171 and the same argument as in Lemma 12.281 the condition (6) of 
Lemma 13.41 can be replaced by the following. For any pure one dimensional sheaf G' ^ which 
admits a strict epimorphism H{E) — » G' in Ay 2 , one has (j) a v (B(E)) -< </> CT v(G"). 

Remark 3.6. Since E E A^ 2 * s concentrated on [—1,0], giving a strict epimorphism E —» G 
as in (o) of Lemma 13.41 is equivalent to giving a surjection of sheaves Ti. (E) — » G. 

As for strict monomorphism F E in (6) of Lemma 13.41 we have the following. 

Lemma 3.7. Let F ^ E be as in (b) of Lemma \3.4\ and C C X as in Lemma \3.^ Then there 
are subsheaves, 

Fx C Oc, F 2 c H°(E), 
such that F is written as an extension, 

— ► F x — > F — > F 2 — ► 0. 

Proof. Let 

— ► F — ► E — > G — ► 0, 
be the exact sequence in A p . Taking cohomology, we obtain the exact sequences of sheaves, 

— >H- l {E) — ► ?r 1 (G) — >F X — >0, (25) 
— > Fx — ► F — ► F 2 — ► 0, (26) 
— > F 2 — ► H°(E) — > H°(G) — > 0. (27) 

Since 7i~ 1 (E) is torsion free and Fx E Coh<i(X), we have 

Fx C w- 1 (s) vv '/H~\E) Oc, 

from the sequence (|25|) . Therefore the sequence (|26p gives the desired extension. □ 
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In Lemma 13.41 let us write the condition 4> a (F) -< (f> a (E) in a simpler way. For F G 
Coh<i(X), let H(j(F) G M be as in Example [2J25] (ii) . 

Lemma 3.8. For a = B + ioj G A(X)c, F G L n (X,{3) and F G Coh<i(X), we have <j) a (F) -< 
<b a (E), (resp. <b a (F) >- (f> a (E),) if and only if one of the following conditions hold. 

• We have the following inequality, 

3Buj 2 , 3Bu> 2 . , , 

/V F < 3-, (resp. Ha(F) > 3-.) 28 

• We have /v(F) = —3Buj 2 /uj 3 and 

uv$(E)p a (F) < v*(E), (resp. lov^(E)^(F) > v§(E).) (29) 

Proof. The condition <^> CT (F) -< 4> a (E) is equivalent to 

ReZ CTm (F) ReZ am (E) 

ImZ am (F) > ImZ CTm (F)' 1 j 

for m S> 0. Since we have 

v B (F) = (0,0,ch 2 (F),ch 3 (F) -£ch 2 (F)), 

the inequality (f3"U|) is equivalent to 

/v(F) _ -ch 3 (F) +Fch 2 (F) m 2 u} 2 B/2-v$(E) 



m mu;ch 2 (F) m 3 oj 3 /6 + muiv^E) ' 

for m 3> by the formula (]12p . The above inequality is equivalent to 

±mV L a (F) + < -uvi(E)fx a (F) + v§(E), 

for m ^> 0. Therefore (j28H or (I29p must be satisfied. □ 

3.2 Evaluations of numerical classes 

In this paragraph, we evaluate the numerical classes of TC~ 1 (E), Ji (E) for E G L°(X, (3). Below 
we fix an ample divisor H on X, and set 

N((3) := {/?' G F 4 (X, Z) I /3' is an effective class with < ■ H < f3 ■ H}. (31) 

The following Lemma 13.91 and Lemma 13.101 seem well-known, but we give the proof for the 
reader's convenience. 

Lemma 3.9. The set N((3) is a finite set. 

Proof. For any ample divisor if' on X, we have 

N(f3) C {(3' G H A (X,R) \ &-H'> 0}. 

Since the ample cone is an open cone, one can find a compact convex polytope in H 4 (X, R) 
which contains M(j3). Therefore J\f(j3) is a finite set. □ 
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Next we set m(f3) G [—00,00) as follows, 

m{(3) := mf{ch 3 (O c ) Ccl satisfies dimC = 1, [C] G M(fi)}. 
Lemma 3.10. We have m((3) > —00. 

Proof. Let Hilb n (X, (3) be the Hilbert scheme of one dimensional subschemes C <Z X with 

(3= [CI n = ch 3 (0 c ). 
If Hilb n _fc(X, (3) is non-empty for k > 0, then we have 

dimHilb n (X,/3) > 3k, 

by adding /c-fioating points to a subscheme C'cl with [C] = f3, n — k = cl^Oc). Then the 
boundedness of Hilb n (X, (3) implies that Hilb n _ fc (X, 0) = for k > 0. □ 

Finally we show the following lemma. 

Lemma 3.11. (i) The image of the map, 

L n (X,{3) 3E^ (ch 2 (?T^)),ch 2 (H (£))) G H 4 (X,Zf 2 , 

is a finite set. 

(ii) For E G L n (X,f3), let (3' = - ch 2 (W" 1 (- B ))- Then we have, 

ch 3 (H°(E)) <n- m{(3') <n- m((3). 
Moreover for a = B + iuj G A(X)c, the image of the map, 

L°(X,j3) (di 3 (H-\E)),ch 3 (H (E))) G Z® 2 , 

is a finite set. 

Proof, (i) For E G L°(X,(3), the equality flU) implies, 

(ch 2 (^- 1 ( J E)),ch 2 (7i (^))) GAA(/3) xM{0). 

Hence Lemma 13.91 yields the result. 

(ii) For E G L n {X,[3), we have TL~ l (E) = Iq where C C X is as in Lemma 13.21 Since 
[C] = (3', we have 

ch 3 (W°(£)) = n-ch 3 (0 c ) 
< n — m(P'). 

Also since /?' G N{f3), we have n — m{j3') < n — m{(3). Suppose that E is a- limit stable. If 
TL°(E) is non-zero, Lemma l3.4( Remark 13.61 and Lemma 13.81 show. 

3Blu 2 ch 3 (H°(£))-Bch 2 (^°(E)) 

-— - < ^(n (e)) = — — • 

Since ch 2 (7Y°(i?)) G M((3), the value ch 3 (TC°(E)) G Z is also bounded below by the above 
inequality and Lemma 13.91 □ 
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3.3 Boundedness of limit stable objects 

In this paragraph, we show the boundedness of limit stable objects, which is relevant to the 
existence of the moduli space. Recall that a set of objects S C D b (X) is bounded if there is a 
finite type C-scheme Q and an object 8 G D b (X x Q) such that any object E G S is isomorphic 
to £ q for some q G Q. We first show the boundedness of some subsets of objects in Coh<i(X). 
For (3 G H A (X, Z) and n G Z, we set 

S n := {£ G Coh<!pO | ch 2 (E) = /3,ch 3 (£) = n}. 

Also let us fix a = B + itu G A(X)c and /i G R. 

Lemma 3.12. (i) The following set of objects is bounded. 

, . j . . ^o-(G) > /x for any surjection \ 

5 n (X,/3, ff ,^ = |£;GS n (X,/3). ^_ Gin Coh ^ l(x) )• 

(raj T/ie following set of objects is bounded. 

aifva \ f r< ^ n u rv\r,AP there is -E G S n (X,(3) and a surjection \ 
S n (X, = |G G Coh<!(X) n^ /2 : E ^ Gm Coh < l(X) and < ^ ) • 

Proo/. (i) For £ G S n (X,P,<r,n), let 

be the Harder-Narasimhan filtration with respect to (P, u;)-twisted semistablity. (Or equiva- 
lently Harder-Narasimhan filtration with respect to the induced stability condition (Coh<i(A A ), Z a ).) 
Note that for F{ = Ei/Ei-%, we have ch2(Pj) G M((3), and the function which sends E G 
S n (X, (3, a, ju) to the number of (P, u;)-twisted semistable factors Z is bounded. By the definition 
of S n (X, (3, a, n), we see 

Ha(E) > fi a {E/E t ) > n, 
for any i. Therefore for each i, we have only finite number of possibilities for the pair, 

(ch(^),ch(£/£ 4 )) £F(I,Qf. 

Hence the possibilities for ch(Pj) G H*(X,Q) is also bounded. On the other hand, the set of 
(B, w)-twisted semistable sheaves F G Coh<i(A A ) with a fixed numerical class is bounded. (See 
for instance [Ml Lemma 3.8].) Therefore S n (X, (3, a, (j,) is bounded. 

(ii) For G G S' n (X, (3, a, fi), note that cl^G) G N{(3). Hence ^ a {G) < fi implies that there 
is n' G R, which depends only on B and (3, such that 

_ ch 3 (G) , 
a;ch 2 (G) 

for any G G S' n (X, f3, a, /i). Since G is a pure sheaf, one can apply [HI Lemma 1.7.9], and 
conclude that S' n (X, (3, a, /j) is bounded. □ 

In the following we show the boundedness of L°(X, (3). 

Proposition 3.13. For a = B + iuj G A(X)c, Z/ie seZ of objects L°(X,(3) is bounded. 
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Proof. It is enough to show the boundedness of the following sets of sheaves, 

{H- 1 (E)\EeL^(X,(3)}, (32) 
{H°(E)\EeL°(X,P)}. (33) 

Also by Lemma T3.1H it is enough to show the boundedness of the above sets of objects satisfying 

(ch 2 (W- 1 (^)),ch 3 (^ 1 (^))) = (-/3W), 
(ch 2 (H°(E)),ch 3 (H (E))) = (l3",n"), 

for fixed numerical classes (/3',n'), (f3",n"). For E G L%(X,P), we have H~ l {E) G I n ,{X,p) 
by Lemma 13.21 in particular the set of sheaves (|3"2"|) is bounded. Here I n > (X, /?') is the Hilbert 
scheme as in the proof of Lemma 13.101 Also Lemma 13.41 Remark 13.61 and Lemma 13.81 show 

H°(E)eS nl ,(X,(3",a,n), 

where \i = —3Buj 2 /ll> s . Therefore by Lemma [3.12l (i). the set of sheaves ([33]) is also bounded. □ 

3.4 Openness of limit stability 

The purpose of this paragraph is to show that the embedding C^(X, 0) C M!q given in ([23j) is 
an open immersion, and complete the proof that C°(X,[3) is an algebraic space of finite type. 
First we see the openness of objects in A v and A\. 

Lemma 3.14. For a variety S and an object £ G D b (X x S), the sets 
S° = {a€S\S.€ A p }, S° = {s € S | £. e A?}, 

are open subsets in S. 

Proof. As in [21 Appendix A, Example 1], the torsion theory (Coh<i(A), Coh>2(A)) defines an 
open stack of torsion theories. Then [21 Theorem A. 3] shows that S° is open. Let us show that 
5° is open in S. We set B(£) to be 

B(£) =KHom(£,Oxxs[2}). 

Then we have B(£) s = B(£ s ). By Lemma [2ZLH S° are written as 

S° = S°n{seS\ B(£) 3 G A p [-i}}, 
s° l/2 = s° n {s G S | B(S) 8 G A p }. 

Therefore the openness of S° follows from the openness of S°. □ 

Next we show the boundedness of destabilizing objects for a family of objects in Ay 2 - 

Suppose that £ G D b (X x S) satisfies £ s G L n (X,0) for any s G S. We set the sets of objects 
V e , V m as follows. 



f p there is s G S and a strict epimorphism 1 

| Coh<x(A) n A 1/2 : £s ^ Gin A P /2 with ^ ^ (£fl) | • 

f there is s £ S and a strict monomorphism "1 

P m = |F G Coh<x(A) n A 1/2 : F ^ fg in A v ^ ^ ^ } j 



We have the following. 
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Lemma 3.15. The sets of objects T> e , V m are bounded. 

Proof. Applying the dualizing functor B, it suffices to show the boundedness of T> e as in Re- 
mark E3J By taking a flattening stratification, we may assume that TL l {£) is flat over S for any 
i. As in Remark 13,61 any object G G V e is obtained as a surjection, 

n°(£ s ) = n°(£) s - G. (34) 

Let (ch2(H (£)s),ch 3 (H°(£)s)) = (P',n'). By (JM]) and LemmaEEl we have 

G G S' n ,(X,p',a,ij,), 

where [i = —3Buj 2 /lo 3 . By Lemma 13.121 (ii). the set of objects T> e is bounded. □ 

Based on the above lemma, we show the following proposition. 

Proposition 3.16. (i) There exist a finite type S-scheme ir e : Q e — > S, Q e G Coh.(X x Q e ), and 
a morphism u e : £q e — > Q e such that 

• For q G Q e , the morphism u e , q : £ q — > £/ e ,cj *s a strict epimorphism in A\i 2 - 

• Any strict epimorphism £ s —» G in Ay 2 f or G £L T> e is isomorphic to u eA for some 
q G vr-^s). 

(ii) There exist a finite type S-scheme Tr m : Q m — > 5, .F m G Coh(X x Qm); anc ^ a morphism 
u m ■ Fm — * £-Qm suc h that 

• For q € Q m , the morphism u m ^ q : J~ m ,q — > £ q is a strict monomorphism in Ay 2 . 

• Any strict monomorphism F w £ s in A\i 2 f or F G *s isomorphic to u m ^ q for some 
q G vr-^s). 

Proof. The proof is essentially same as in [33^ Proposition 3.17], so we only give the outline 
of the construction of Q m . Since V m is bounded, there is a C-scheme of finite type Q and 
T G Coh(A" x Q), flat over Q, such that any F G T> m is isomorphic to T q for some q G Q. 
We may assume that <f> a {T q } >z 4>a(£s) and .F 9 is a pure one dimensional sheaf for any q G Q. 
Arguing as in |33|. Proposition 3.17], there is an affine scheme of finite type Q' and a morphism 
Q' —* Q x S such that 

• Q' — > Q x 5 is bijective on closed points. 

• There exists a locally free sheaf U on Q' such that the functor 

(T -> QO i — ► H (Rg T *RWom(^ r ,^ T )) G Coh(T), 

is represented by the affine bundle Y(U) — > Q', where gy: X x T -» T is the projection. 

Here ^r, £t are obtained by the base changes of .F, 5 for the following morphisms respectively, 

T -> Q' -> Q x 5 ^ Q, T^Q'^QxS^S, 

and pi, p2 are projections. Let u: J--y(u) ~^ £v(u) De the universal morphism and take the 
distinguished triangle, 
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Note that u q : Jy(u),q ~^ £y(U),q is a strict monomorphism in A\, % if and only if Q q G Ay 2 - We 
construct Q m as 

Q m :={q€V(U)\g q €j% /2 }. 

Then Q m is an open subscheme of V(ZY) by Lemma I3.14| in particular it is of finite type. By 
the construction, 

7i"m • Qm — ► S, T m := T^(U)\Q m G Coh(X x Q m ), u m := u\Q m , 

satisfy the desired property. □ 

Remark 3.17. By the construction and Lemma 13.21 the object £ s G L n {X,0) is cr-limit stable 
if and only if 

a i vr e (Q e ) U7r m (Q 

m I ■ 

Here we collect some well-known lemmas on a family of objects in D b (X). For the lack of 
reference, we also put the proofs. Let R be a discrete valuation ring and K a quotient field of 
R. We denote by t G R the uniformizing parameter, and o G Speci? the closed point. We set 
X R = X x Speci?, X K = X x Specif. 

Lemma 3.18. Take J 7 , £ G Z) fe (X x Speci?) and a non-zero morphism f : — ► *n 
T/ten ffoene ismSZ sue/t i/iai 

• T/ie morphism t m f: Tk £k extends to a morphism t m f : .F — > £ . 

• The induced morphism t m f\xx{o} '■ 3~o ~~ > £o is non-zero. 
Proof. Since Hom(.F, £) is a finitely generated i?-module and 

Homx fl (T, £)®rK = Homx K {T k ,£k), 

there is m G Z such that t m f extends to T — ► £ and t m ~ 1 f does not extend to T — > We have 
the exact sequence in CoIi(Xr), 

— >£ 2^ S — >£ — > 0. 

The above sequence shows that if t m f\ X x{o} is zero, then t m f factors though T — > £ ^> £, which 
gives an extension of t m ~ 1 f. Therefore t m f\xx{o} 1S non-zero. □ 

Let T be a (not necessary projective) smooth curve with a closed point o G T. We set 
T° = T\{o}. 

Lemma 3.19. Suppose that T G Coh(X x T°) is flat over T° and Tt G Coh<i(X) for any 
t G T°. 

(i) Assume moreover that Tt is (B,uj) -twisted semistable for any t G T° . Then there is 
T G Coh(X x T), which is fiat over T, such that T\xxT° — T and T\xx{o} is a ^so [B, un- 
twisted semistable. 

(ii) There is an open subset T C T° and a filtration of flat sheaves over T , 

T -pi a C J~rpl o -y G * " * d J~rp' o ^ T pi , 

such that for any t G T'°, the induced filtration of T is a Harder- Narasimhan filtration with 
respect to (B , u) -twisted semistability. 
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Proof, (i) For a = B + iu, let us consider the induced stability condition (Z a , Coh<x(X)). 
Using [33, Proposition 2.8], we can assume that B and u) are defined over Q. After applying some 
element of C to Stab(L> b (Coh<i(X))), we obtain a stability condition (Z',A') on Z? 6 (Coh<i(X)) 
such that A' is a noetherian abelian category and Z'{Ts) G M<o for any s G £7. (See [331 Remark 
2.7].) Then we can apply [H Theorem 4.1.1] and conclude the result. 

(ii) If B = 0, this is shown in |14} Theorem 2.3.2]. The twisted case is similarly discussed 
and we leave it to the reader. □ 

Now we are ready to show the following theorem. 

Theorem 3.20. The embedding £^(X, /3) C A4q is an open immersion, and C^(X,P) is a 
separated algebraic space of finite type over C. 

Proof. First we show that C^(X, 0) C is an open immersion. For a variety S, let us take 
a S'-valued point, £ G M.Q t (S). Suppose that £ s G L^(X,P) for some point s G S. We want to 
show that there exists a Zariski open subset s G U C S such that £ s i G -^(^, /3) for any s' G £7. 
Applying Lemma f3. 141 we may assume £ s i G L n (X,f3) for any s' G 5. Let us construct 

ir e :Q e ^S, Q e G Coh(X x Q e ), u e : £ Qe -> £ e , 

: Qm * <5> 3~ m E Coh(JT x Q m ), u rn : T m ► £Q m , 

as in Proposition 13.161 Noting Remark 13.171 it is enough to show 

S i TT e (Qe) U 7T m (Q m ). 

Let us show s ^ vr m (Q m ). The proof of s ^ vr e (Q e ) is similar. Suppose by a contradiction that 
•s G TT m (Q m ). Then we can find a smooth curve T with a closed point o G T and a morphism 
p : T — > S such that p(o) = s and there is a commutative diagram, 

T" 3 *" Qm 



where T° = T \ {o}. By pulling back JT m g Coh(X x Q m ) to I x T°, we obtain the object 
3~m,T° G Coh(A x T°) and a morphism, 

«m,T° : Fm,T° ► £t° , 

such that 4> a {!Fm,t) h <t><r(£t) for any t G T°. Applying Lemma T3.19I (ii), we may assume that 
J- m ,t is (B, uj) -twisted semistable for any t G T°. Then Lemma [3. 191 (i) shows that there is a flat 
family of (B, w)-twisted semistable sheaves, 

f m G Coh(X x T), 

which extends J- m ,T°- Applying Lemma 13.181 (i) for R = Ot > we obtain a non-zero morphism, 

•F m,o > £s- (35) 

Note that T m ^ is er-limit semistable, £ s is er-limit stable, and 4>(r{^m,o) ^ 0cr(£s)- This implies 
that (t>a{F mj0 ) = <Pa{£s)i the object £ s is one of the Jordan-Holder factors of JF m o , and the 
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morphism (1350 is surjective in A p . However in this case £ s must be an object in Coh<i(X) as 
we remarked in Remark l2.13[ which contradicts that £ s G L n (X,{3). Therefore s £ 7r e (Q e ) holds. 

Now we have proved jC^(X,f3) C M.'q is an open immersion, hence C°(X, 0) is an algebraic 
space of locally finite type. Moreover L^(X,P) is bounded by Lemma 13.131 which implies that 
C°(X,f3) is in fact of finite type. 

Finally let us show that C^(X,/3) is separated using valuative criterion. Let R, K , t G R 
and o G Speci? be as in Lemma T3. 181 Take two R- valued points of C^(X, (3), £1,82 G D b (Xji). 
Suppose that there is an isomorphism in D b (Xx), 

f- £i,k — * £2,K- 

quotient field of R. By the valuative criterion, it is enough to show that t m f extends to an 
isomorphism £\ — * £2 for some m G Z. By Lemma l3.18| there is m G Z and a morphism 

/: £1 - £2 

which extends t m f and the induced morphism f a : £i )0 — ► £2,0 is non-zero. Since £i jD and £"2,0 are 
both (T-limit stable objects with the same numerical classes, the morphism f D is an isomorphism. 
Hence / is also an isomorphism. □ 

Remark 3.21. In [33], the author used the result of [TJ Proposition 3.5.3] to show the openness 
of stability for the case of K3 surfaces. In the situation of our paper, the relevant abelian category 
A p is not noetherian which prevents us to use the result of pp. Instead we have used Lemma f3,18l 
Lemma 13.191 to show the openness. 

Remark 3.22. It seems likely that C^(X,/3) is a projective variety for a generic choice of a, 
which we are unable to prove at this time. We do not how to construct the moduli space as a 
GIT quotient. Also the main technical difficulty to show the properness is that we are unable to 
use extension results of a family of objects as in [TJ Theorem 4.1.1], since A v is not noetherian 
again. 

4 Counting invariants of limit stable objects 

In this section, we again assume that X is a projective Calabi-Yau 3-fold. The purpose of this 
section is to construct virtual counting of u-limit stable objects, and study their properties. 

4.1 Definitions of counting invariants 

For a G A(X) C , (3 G H 4 (X,Z) and n G Z, let C°(X,/3) be the algebraic space constructed in 
Theorem l3.20i In this paragraph, we give the definition of the counting invariant of cr-limit stable 
objects L nt p(a) G Z using £°(X,[3). Since we are unable to conclude that C^{X,j3) is proper, 
the integration of virtual classes does not make sense. Instead we use K. Behrend's constructible 
function [3] to define counting invariants. Recall that Behrend [JJ constructs on any scheme M, 
(more generally M is a Deligne Mumford stack,) a canonical constructible function, 

v M : M — ► Z, 

which depends only on the scheme structure of M. If M is smooth, vm is given by vm{p) = 
(— l) dimM . Moreover if M is proper and carries a symmetric perfect obstruction theory, one has 

r(M) = J2ne(vM 1 (n)), 



25 



where $ YU (M) is the integration over the virtual cycle, and e(*) is the euler number. In our 
situation, let 

v L : C%X,P) — >Z, 

be Behrend's constructible function. 

Definition 4.1. We define the invariant L n .p(a) € Z by the formula, 

Note that since an algebraic space of finite type is stratified by affine schemes of finite type, 
its euler number makes sense. 

Remark 4.2. Suppose that C^(X,P) is a proper algebraic space. Then by the same argument 
as in [271 Lemma 2.10] and [13], there is a virtual fundamental class, 

K(I,/3)fe4 (^(I^)). 

By the above argument, our invariant L n p(a) coincides with the integration over the virtual 
class, 



Ln,p( a ) = / 1 - 

The purpose of this section is to relate the invariants L n a((j) to the invariants of stable pairs 
on a Calabi-Yau 3-fold introduced by Pandharipande and Thomas [27]. Let us recall the notion 
of stable pairs. 

Definition 4.3. |27J A stable pair consists of data (F,s), 

a: O x ^F, 

where F is a pure one dimensional sheaf and s is a morphism satisfying 

dimCoker(s) = 0. 

Given a stable pair (F, s), we can associate the two term complex, 

/• = (Ox — » F) G D b (X), (36) 

where F is located in degree zero. As we mentioned in Remark 13. 1[ if F satisfies 

ch 2 (F) = (3, ch 3 (F) = n, (37) 

we have I* € L n (X, (3). By abuse of notation, we also call the two term complexes (|36p as stable 
pairs. In [27], the moduli space of stable pairs (F, s) satisfying the condition ((371) is constructed 
as a projective variety, and denoted by P n (X, 0). The obstruction theory on P n (X, (3) is obtained 
from the deformation theory of the two term complexes J* = (Ox —> F). 

Definition 4.4. [27J A PT-invariant P n>/3 G Z is defined by 

P n ,f3= I l = Y j ne(vp 1 (n))eZ. 

Here vp\ P n (X,f3) — > Z is Behrend's constructible function. 
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4.2 Limit stable objects and stable pairs 

The purpose of this paragraph is to investigate the relationship between L n ^{a) and P n ,p- First 
we show the following lemma. 

Lemma 4.5. An object E G L n (X,f3) (cf. 120\) ) is isomorphic to a stable pair 136\) if and only 
if7i°(E) is zero dimensional. 

Proof. Only if part is obvious, so we show the if part. For an object E G L n (X, /?), suppose that 
H°(E) is zero dimensional. Applying Hom(*, O x [l]) for the triangle H~ l {E)[l] —>£'—> H°(E), 
we obtain the exact sequence, 

Rom(H°(E),O x [l}) -> Rom(E,O x [l}) -> Hom^ 1 ^), O x ) -» Hom(H°(E), O x [2\). 

Since TiP(E) is zero dimensional, the Serre duality implies 

Rom(H°(E),O x [j]) = H 3 ~i(X,H°(E)) = 0, 

for j = 1,2. Hence we have the isomorphism 

Hom(£,e> x [l]) ^Rom(n- 1 (E),O x )- 

By Lemma 13.21 we have TC^ 1 (E) = Iq for a one dimensional subscheme C C X. Therefore 
there is a morphism u: E — > corresponding to the inclusion Iq C Ox- Let us take the 

distinguished triangle, 

O x F ^ E ^ O x [l}. 

It is enough to show that F is a pure one dimensional sheaf. Since E G L n (X, 0) , it is obvious 
that F G Coh<i(X). For a closed point x G X, we have the exact sequence, 

= Bom(O x ,O x ) — ► Hom(O x ,F) — » Hom(O x> £)• 

Since E 1 G «^i/2> we have Hom(O x , E) = 0, hence Hom(O x ,F) = 0. Therefore F is a pure 
sheaf. □ 

In the following, we focus on a G A{X^ of the form, 

a = kuo + iuJ, k eR, (38) 

and see how C^(X,f3) and L n ^(a) vary under change of G M. The advantage of setting a as 
(1381) is as follows. 



Lemma 4.6. For F G Coh<i(X) and E G L n (X,(3), the condition 4> a (P) d: <t>a(E) (resp. 
<t>cr (F) h 4>a{E)) implies 

k<-^l, {res P .k>-^l) (39) 
Proof. If a = kuj + iu, we have 

3Buj 2 

fJ, a (F) = Hiu(F) - k, — = -3k, 

for B = ku. Hence the inequality (|39|) follows from the same argument as in Lemma 13.81 □ 
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We also note that (ku, w)-twisted (semi)stable sheaves coincide with w-Gieseker (semi)stable 
sheaves. 

We set \x n $ £ Q as follows, 

( n — m(B") 

H n ,p = max <^ -pi : + 0, 0' G M(0) and = + /3" 

Since N(j3) is a finite set, we have \x n ^ < oo. The following is the main result of this section. 
Theorem 4.7. Lei a = ku> + im for k £ R. We /icwe, 

= P n (X,/3), L n>/ ,(<7) = P n>/3 , i/fc < 
C%(X,0) = P-n(X,P), L nfi (a) = P_ n>/3 , i/fc > /X- n ,^/2. 

Proof. First assume A; < —fi n ^/2 and take -E € L°(X,[3). (cf. (|2ip .) In order to show E is 
isomorphic to a stable pair (f36l) . it suffices to check that H°(E) is zero dimensional by Lemma H3J 
Suppose by a contradiction that 7i°(E) is one dimensional, and set 

= ch 2 (H°(E)) + 0, 0' = - c^iH-HE)). 

By Lemma 13.111 we have 

ch 3 (H°(E)) <n-m{0'). 
Since E is a-limit stable, we must have 4> a (E) -< (j) a (fi°(E)). Lemma 14.61 implies that 

HiUn°(E)) _ _ch3(W%B)) n - m(/?") /yg 
2 2w/?' - 2w/3' - 2 ' 

This contradicts that k < —/jj n p/2, hence E is isomorphic to a stable pair. 

Conversely take a stable pair E = (Ox —> F) G P n (X,/3). We have to check the conditions 
(a), (6) in Lemma 13.41 Let E -» G be a strict epimorphism as in (a) in Lemma 13.41 By Re- 
m ark [3*1)1 G is obtained as a surjection of sheaves 7i°(E) -» G. Since Ti (E) is zero dimensional, 
G is also a zero dimensional sheaf, hence G £ provided G / 0. This means that (a) does 
not occur, so it is enough to check (6) in Lemma 13.41 

Let F' ^ E be & strict monomorphism as in (b) in Lemma |3.4( and set = ch.2(F'). Let 
C C X be a one dimensional subscheme with 7^ _1 (i?) = Iq. We can take subsheaves F\ C Oc, 
i*2 C TiP(E) as in Lemma 13.71 such that -F' is written as an extension 

— ► Fi — >F' — > F 2 — ► 0. 

Since Oc/F\ = Oc for a subscheme C C C, we have 

C h 3 (F!) =ch 3 (e>c)-ch 3 (0C') 

<ch 3 (O c )-m(0'), 

where 0' = cb_2(0c")- Hence we have 

ch 3 (F') = ch 3 (F 1 ) + ch 3 (F 2 ) 

<di 3 (O c ) + ch 3 (ri°(E))-m(0') 
= n-m(0'). 
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We obtain the inequality, 

. . Vn,f3 ^ n - m((3") m^jF") 

which implies cf) a (F') -< (j) a {E) by Lemma 14.61 Therefore (6) in Lemma 13.41 holds, hence E G 
K(X,/3). 

By the above arguments, the set of C- valued points of C^(X, (5) and P n {X, (3) are identified. 
Therefore we have the isomorphism of the moduli spaces, 

CZ(X,P)*P n (X,P), 

since both are open algebraic subspaces of Mq 1 . In particular L n ^{a) = P n< p follows. When 

k > / u_ n>( g/2, we have 

L n ,p{°) = L- n> p(<r ) = P-n,/3, 

by applying the dualizing functor ID and using Lemma 12.281 □ 

Remark 4.8. The wall-crossing phenomena for stable pairs is also studied in Bayer's polynomial 
stability conditions [HJ Paragraph 6.2]. However our wall-crossing is crucially different from 
Bayer's wall-crossing. In fact the complexified ample cone A(X)q, the stability parameter in 
our stability conditions, behaves itself as a wall in the wall-crossing of Bayer [SJ Paragraph 6.2]. 



4.3 Wall-crossing phenomena of limit stable objects 

Let a = kuj + iuj be as in the previous paragraph. In this paragraph, we investigate how o- 
limit stable objects vary under change of k 6 R. As we have seen in Theorem 14. 7} limit stable 
objects coincide with stable pairs for k <C 0, and the dual of stable pairs for k 3> 0. We look at 
the wall-crossing phenomena more closely, which hopefully might be helpful for the rationality 
conjecture of the generating functions of PT-invariants, proposed in |27j . 
For an effective class (3 G H 4 (X,Z), we set S{f$) C R as 

S{(3) := j-^- : 0^7 eM(/3),m ezl C R, 
1 2^7 J 

where N{(3) is introduced in (f3T|) . Note that S(/3) is a discrete subset in R because M{(3) is a 
finite set. In the following, we see that S{(3) behaves as the set of walls. 

Proposition 4.9. Let C C R\«S(/?) be one of the connected components. For k, k' E C, we have 

£^X,P)=JC^(X,0), 
where a = koo + ioj, a' = k'u + too. In particular the function, 

R 3 k i — ► L n fi{ku) + iuo) G Z, 

zs constant on C. 

Proof. For E G L^(X, assume that E ^ L^(X,f3). Then at least one of the conditions (a) 
or (6) in Lemma 13.21 does not hold. Suppose that (a) does not hold and let E -» G be a strict 
epimorphism with G G Coh<i(X), which destablizes E. Then Lemma [3.81 and Lemma [4.61 show . 

k' < (40) 
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Since cl^G) G N(j3) by Remark [3.61 the right hand side of (jlUI) is an element of S{j3). Therefore 
k satisfies the inequality k < — ^,(G)/2, which implies 4> a {E) y <f> a {G). This contradicts that 

The case that the condition (b) in Lemma 13.21 does not hold is similarly discussed, noting 
Lemma 13.71 which shows that cl^i 7 ) G N(/3) for destablizing monomorphism F E. □ 

Next we investigate the wall-crossing phenomena at some point — /u/2 G S{(3). Let C_, 
be connected components in R \ <S(/3) such that 



C_ c R<_ M/2 , C+ c %_ M/2 , C_ n C, 
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Let us take k- G C_, fc + G C + and fco = — fJ,/2. We set a* = k*u + iuj for * = ±, 0. We have the 
following proposition. 

Proposition 4.10. (i) Assume that E G L n ~(X,[3) is not a+ -limit stable. Then there is an 
exact sequence in A\i 2 , 

— ► F — >E — ► G — ► 0, (41) 

such that F is co-Gieseker semistable sheaf with [i^ = [i and G is a+-limit stable with 4> a+ {F) y 
<p a+ (G). i.e. Iji41\ ) is a Harder- Narasimhan filtration in a + . The object G is also a^-limit stable, 
(ii) Assume that E G Ln + (X, 0) is not o~--limit stable. Then there is an exact sequence in 

A v 

— ► G — >E — >F — ► 0, (42) 

such that F is to-Gieseker semistable sheaf with fj,^ = fi and G is a --limit stable with 4> (T _ (G) y 
(p a _(F). i.e. |^^| ) is a Harder- Narasimhan filtration in <r_. The object G is also a^-limit stable. 

Proof. The proof of (ii) is identical to (i) , so we only show (i) . Assume that E G L n ~ (X, (3) is 
not <r + -limit stable. By Lemma 13.41 we have one of the two possibilities. 

(a') There is a non-zero pure one dimensional sheaf G which admits a strict epimorphism 
E -» G in A P 1/2 with <f> a+ (E) y <t>a+(G). 

(&') There is a non-zero pure one dimensional sheaf F which admits a strict monomorphism 
F E in A P 1/2 with ^ + {F) y <f> a+ (E). 

Suppose that (a') occurs. By Lemma l4~6l we see that 

k _ < k+ < 

+ ~ 2 

Therefore we also have 4> a _(E) y (j> a _(G), which contradicts that E G L n ~(X,(3). Hence (b 1 ) 
occurs. Let F E be as in (&'). Since Ay 2 is of finite length, one can take such F to be 

maximal, i.e. there is no non-trivial strict monomorphism F F E such that F is also pure 
one dimensional sheaf with (j> a+ (F) y 4> <J+ {E). Since 4> <J+ {F) y cp rr+ (E) and (f> a _(F) ~< (j) a _(E), 
we have 

k. < -f^- < k + . 
Since cl^i 7 ) G Af((3), we have —Hi UJ {F)/2 G S((3). Therefore we have 

/TTT\ 7 ^ilo{E) 

Viw{F) = (J,, k + > — , 
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which implies <pa + {F) >~ <Pcr + {F). In order to show F is a w-Gieseker semistable sheaf, it is 
enough to show that F is <7+-limit semistable. Let F' C F be a strict monomorphism in Ay 2 . 
Note that F' is also pure one dimensional sheaf. If 4> a+ {F') y cp a+ (F), then (p a+ (F r ) y cp a+ (E), 
hence ^{F') = fi by the same argument as above. It follows that F is <r + -limit semistable. 
Let us take the exact sequence in Ay 2 , 

— > F — ► E — >G — ► 0. 

We want to show that G is delimit stable for * = +,0. We show the case of * = 0, as the proof 
for the other case is similar. Suppose the contrary. By the same argument as above, there is a 
strict monomorphism F" > G in Ay 2 such that <t> ao (F") y <j) ao (G). We have 

k + > k > i , 

by Lemma 14.61 which in turn implies 4> a+ (F") y <f) a+ {E). Let F'" be the kernel of the composi- 
tion of the strict epimorphisms 

E -» G -» GIF" . 

Then we have the non-trivial strict monomorphism, F <^-> F" >^-> £J with <p (T+ (F") y <f) a (E), 
which is a contradiction since F F is maximal. □ 

Let Zj T be the generating series, 

Z| T (g) = ^P ni/3 g«GQ((g)). (43) 

nez 

In \17\ Conjecture 3.2], Pandharipande and Thomas conjecture that the generating series (|43p is a 
rational function of q, invariant under q \—> 1/q. This conjecture (rationality conjecture) is solved 
when f3 is an irreducible curve class case in [29], and the crucial point is to find a relationship 
between P n a and P_ n a. By Theorem 14. 1\ it is possible to obtain such a relationship in a general 
situation by establishing a wall-crossing formula of our invariants L n ^(a). Suppose for instance 
that any F £ Coh<i(X) which appears in the sequence (|4ip . (|42|) is in fact u-stable, and satisfies 

(ch 2 (F),ch 3 (F)) = (P',n'). (44) 

Let Cr C £n~(X,/3) be the unstable locus in a + -limit stability, C + C £n + (X,f3) the similar 
locus, and M n /(X, (3') the moduli space of w-Gieseker stable sheaf F £ Coh<i(X) satisfying 
). The destabilizing sequences (jlTj) . (|4"2"1) yield the following diagram, 

£" _ „ £+ 

M n ,(X,pi)x£°°(X,[3"). 

Here /?' + (3" = (3 and n' + n" = n. From the above diagram, one might expect the formula 
something like 

L n>/3 {a-) - L n> p(a + ) = (ft Ext^G?, F) - (t Ext 1 ^, G))N n , tP ,L nl ,^{a Q ), (45) 
where N n i w is the virtual counting, 

JV«',/3'= / l€Z. 
J[M„,(X,/3')] vir 
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We expect that the contribution of the term ((J Ext (G,F) — jJExt (F,G)) is given by 

(-ir'- 1 X (F,G) = (-ir'- 1 n', 

so the right hand side of (jl5l) should be (— l) n -1 n' N n i $ L n n $n (o" ) . 

In case a destabilizing sheaf F G Coh<i(X) which appears in (|41j) or (I42p is strictly 
semistable, the construction of the invariant N n i g> is problematic. It seems that Joyce's motivic 
invariants of moduli stacks [181 Definition 3.18] are relevant for this problem, although Joyce's 
invariants are not deformation invariant as they do not involve virtual classes. Hopefully it is 
possible to involve virtual classes (probably using Behrend's constructible function [3], ) and the 
following wall-crossing formula should hold. 

Conjecture 4.11. There is a virtual counting of tu-Gieseker semistable sheaves F € Coh<i(X) 
with (cli2(-F), ch.^{F)) = (/?', n'), denoted by N n i ni G Q, such that 

L n>p {a-) - L n>p {a + ) = ^(-l) n '~ VA^L^^o). (46) 

Here in the above sum, (/?', n'), (/?", n") must satisfy (3' + (3" = f3, n' + n" = n and n' /u)(3' = fi. 

Note that a term in the right hand side of (|46p is non-zero only if f3',{3" £ N(f3), so there 
are only finite number of non-zero terms. Also for a non-zero term in (]46p . the class (3" is 
smaller than /?, i.e. < 0" ■ H < (3 ■ H for an ample divisor H. Hence if Conjecture 14.111 
is true, we can describe how the invariants L n ^{a) vary under change of a inductively on /3, 
and eventually provides a relationship between P n $ and P-n.p- We expect this relationship will 
show the rationality conjecture of the generating series (|43|) . 

In the next paper [35], we will proceed this idea further using D. Joyce's work |18] on the 
wall-crossing formula of counting invariants of semistable objects in abelian categories. It will 
turn out in [35] that the similar rationality property holds for the generating functions of euler 
numbers of the moduli spaces of stable pairs, using the results in this paper. We remark that 
Joyce's work is applied for the invariants without virtual fundamental cycles. However by the 
recent progress in this field |22j , [19] , [20] , we guess that the similar wall-crossing formula should 
hold after involving virtual classes. At this moment, the works [22], [20] are not enough to 
conclude the rationality conjecture. (The work [22] assumes \22\ Conjecture 4] to show the 
main result [221 Theorem 8], and the result of [20] is only applied for counting invariants of 
coherent sheaves, not for those of objects in the derived category.) Finally we mention that 
T. Bridgeland [TU] proved the rationality conjecture assuming the main result of Kontsevich- 
Soibelman [221 Theorem 8], without using any notion of stability conditions. 



5 Examples 

In this section, we see the wall-crossing phenomena of limit stable objects in several examples. 



5.1 (3 is an irreducible curve class 

Suppose that (3 G H 4 (X,Z) is an irreducible class, i.e. (3 = [C] for an irreducible and reduced 
curve C C X. For a = kuj + iu, we have 



P n (X,(3) iffc<-^, 
P-n(X,f3) iffc>-^. 
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Note that fi n ^ = n/u/3 in this case, so Theorem 14.71 yields the above result. The formula (|46f) 
becomes 

Pn.fl ~ P~n,f3 = (-lT^nN^, 

which is proved in [23 Proposition 2.2]. Hence Conjecture 14. Ill is true in this case. 
5.2 (3 is a reducible curve class 

Suppose that there are smooth rational curves C±, C 2 on X such that 

a Ci/x ^o q (-i)® 2 , p = [c 1 ] + [c 2 ], dnc 2 = M, 

where C\ n C 2 is the scheme theoretic intersection. Let C = C\ U C 2 , di = uj ■ Ci and assume 
that d\ > d 2 > 0. This is possible if C\ and C 2 determine linearly independent homology classes 
in H 2 (X,R). As for £f(X,/3), we have the following, 



P 1 (X,f3)^SpecC iik<- w ^ ry 
P_i(Jf,/3)=0 if fc >_ 3 ^ sy . 



In fact /ii j( g = l/(di + cfo) and H~\ n = —l/{d\ + efe) in this case, so we can apply Theorem 14.71 
The set of stable pairs E € A p with (ch 2 (-E), chs(E)) = (1,(3) consists of one element {Jc[l]}, 
and we can easily compute 

Ext^(/ c [l],/ c [l])=0. 

Hence scheme theoretically P\{X,(3) is isomorphic to SpecC. If k > — l/2(di + d 2 ), then the 
exact sequence in A p , 

0^O c ^ I C [1] — » O x [l] — 0, 
destabilizes /c[l]- According to Proposition 14. 1U| we might obtain stable object as an extension, 

— ► O x [1] — ► £ — ► Oc — ► 0. 

However since Ext^(Oc, = H l {Oc) = 0, the above sequence splits, so E is not cr-limit 

stable. In fact we can check that P_i(A, /3) is empty in this case. The counting invariants are 
as follows, 



L lA a ) 



1 tfk< 2 (di+d 8 )' 
lf k > ~2(d 1 +d 2 )- 



The formula (|46p is easily checked to hold in this case. 

Next let us investigate C^iX,^). The result is as follows. 

( P 2 (X,f3)^C i£k<-^, 

In this case, we have \i 2 $ = l/d 2 and M-2,/3 = — 2/(^1 + ^2)- Also giving a point of P2(X,[3) is 
equivalent to choosing a closed point of C. By Theorem 14. 71 together with some more arguments, 
we see 

£ 2 r (X,(3)=P 2 (X,(3)^C, 

for k < -l/2d 2 . 
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Suppose that 

1 , 1 

< k < 



2d 2 d\ + d 2 

Any stable pair E E P2(X,/3) admits an exact sequence in A p , 

— / c [l] -^E^O x — >0, (47) 

for x E C. If x E C*2, then we have the exact sequence in A p , 

0^O C2 ^E^ I Cl [l] — 0, 

which destabilizes E 1 . In fact E E -4 P given in (|47p is cr-limit stable if and only if x ^ C 2 . Hence 
C\ \ {p} = C is embedded into C^iX,^). It is compactifled by adding a point corresponding to 
the (unique) extension, 

— ► I Cl [1] — ► E' — > C2 — » 0. (48) 

One can check that 

L-(X,/3) = (C 1 \M)U{ J B'}. 

These objects are also obtained as two term complexes Ox —> F, where F E Coh<i(X) is a 
unique non-trivial extension, 

— ► Oq\ — > F — > Oc 2 — ► 0. (49) 

From these observations, we see that 

C%{X,0) = F(H°(F)) = F 1 . 

Finally suppose that k > —\/{d\ + d 2 ). Then for a two term complex E = (Ox —> F) where F 
is as in (JUJ), the exact sequence in A p , 



— > F — > E — ► O x [l] — ► 0, 

destabilizes E. Also there are no non-trivial extensions, 

— ► O x [l] — >E' — > F — ► 0, 

because Hom(F, O x [2]) = H X (F) = 0. In this case, one can check that £f (X, (5) = P- 2 (X, (3) 
0. For the counting invariants, we obtain 

L % p{0) = I -2 if - ^ < k < -gjigj, 

The formula ()46p also holds in this case. For instance, take 

1 1 1 1 

«- < — TT^-i ^0 = ~7TTi ~7TT < k+ < 



2d 2 2d 2 2d 2 t d! + d 2 

A term in the sum of (|46p is non-zero only if = [C 2 ], n' = 1, f3" = [C{\ and n" = 1. We have 
N n ',(}' = 1) and -^n",/3"( <T o) = 1- We can check (|4*6l) as follows, 

L 2)/3 (a_) - L 2i/3 (<t + ) = -1 - (-2) = 1, 

(-1) WiV U c 2] L li[Cl] (<7 ) = (-1) 1 - 1 ! -1 = 1. 
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5.3 (3 is a multiple curve class 

Let C C X be a smooth rational curve with 



N c /x = O c (-ir 2 , (3 = 2[C\. 



"c/x 

Let a = kuj + iui and set d = lo ■ C. For C^(X, (3), we have the following, 

PgP^/^P 1 if < -|, 

P^(X,(3)=® if A: > — g. 



In this case, we have = 2/d and ^-3,/? = — 2/d so Theorem 14,71 is applied. We have 
P 3 (X,/3) = P 1 for k < -l/d by (27J Section 4]. If k > -1/d, we have the exact sequence in A p 
for EeP 3 (X,(3), 

— » O c (l) — £ — / c [l] — 0, 
which destabilizes E. Since Hom(C(7(l), /c[2]) = 0, there is no non-trivial extension, 

— ► J C [1] — ► E' — ► C (1) — » 0, 

and in fact /?) = P- 3 (X,f3) = in this case. The counting invariants are 

/ -2 if k<-\, 



The formula f|46[) also holds in this case. 
In the same way, L\(X, /3) is as follows, 



P 4 (*,/3) iffc<-^, 
££(X,/3) = «{ SpecC if _^ 

P_ 4 (X,/3)=0 iffc>-£. 

Note that fi^p = 3/d and ^-A.fi = —2/d in this case. If —3/2d < k < —l/d, the sequence 

— > O c (2) — » £ — J c [l] — 0, 
destabilizes S -P 4 (^, /3). Instead the unique non-trivial extension 

— > I C [1] — > E' — > O c (2) — 0, 

becomes a-limit stable. The object E7' is isomorphic to a two term complex Ox — * Oc(l)® 2 j 
and the sequence 

— » O c (l)® 2 — ► # — » Ox[l] — 0, 

destabilizes if jfe > -l/d. We have Hom(O c (l)® 2 , O x [2]) = 0, and in fact C%(X,p) = 
P-^{X,0) = for k > —l/d. The counting invariants are as follows, 

4 if/c<-^, 



L 4j/3 (a) = <; 1 if - i d < k < -\, 
iffc>-i 

For P^p = 4, see U3 Section 4]. If k- < -3/2d, k = -3/2d and -3/2d < k + < -l/d, one can 
check that (I46|) holds. On the other hand, the formula (I46D is problematic if 

3 , 1 , 1 i 1 

-Yd <k - < -d^ ko = -d> k+> ~d- 
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Since Oc(l)® 2 is not w-Gieseker stable, we do not how to define N^ 2 [C\- I n this case, N^ 2 [C\ 
should be defined by Joyce's invariant [18, Definition 3.18] after involving virtual classes. For in- 
stance let us ignore virtual classes. By definition, Joyce's invariant N^ 2 [C] is the "euler number" 
of the following "virtual" stack, 

[SpecC/GL(2,C)] - - [SpecC/(A 1 x G^)] , 

which results N^ 2 [C] = —1/4. We have 

LiA a -) ~ L iA a +) = 1-0=1, 
(-l)4-i 4 7V 4)2[c] L ,o(ao) = -1 • 4 • (-1/4) -1 = 1, 

as desired. 
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